Exhibits to Amplify’s July 24, 2019 letter to DPI 


A. Evaluation Consensus votes summary 

(presented at protest meeting, derived from July 12 public records production) 

B. DIT cancellation approval memo 

(presented at protest meeting, from DIT records production) 

C. March 21 memo from DPI to DIT requesting cancellation 
(copy provided by DPI at protest meeting) 

D. Evaluation Criteria changes 

(presented at protest meting derived from July 12 public records production) 

E. May 23, 2019 Draft DPI Award memo (p. 5 notes DPI only used “evaluation factors 
deemed appropriate”, this text was removed from final memo) 

F. SOC 2 Type II Issues 



Evaluation Consensus 


EXHIBIT A 


Requisition No. RQ20680730 (Meeting Notes, November 19-20, 2018) 


Committee Votes 


Amplify Istation 


REQUIREMENTS 


Dyslexia Screening (BS9) 


Developmentally Appropriate (BS8) 


Information Security Audit SOC2 (TS35) 


Reading Assess To inform instruction (BS1) 


Standards Alignment (BS17) 


Progress Monitoring (BS6) 


Diagnostic Assessment (BS12) 


Project Management /WBS (PMS1) 


Educators/Students Use Devices (BS10) 


Minimize Time (BS15) 


Parent Communications (RS4) 


Formative Assessments (BS11) 


Assessment Validity & Reliability (BS3) 


Benchmarking (BS24) 


Identifying Intervention / Enrichment (BS5) 


EVAAS(BS19) 


Reporting Capabilities (RS3) 


NC Security Manual (TS6) 


11 

11 

11 

11 

7 

8 
10 
11 
11 
7 
7 

9 

10 
11 
11 
11 
11 
0 


YES VOTES 


ZERO YES VOTES 


0 

0 

0 

0 

0 

1 

2 

2 

3 

3 

3 

7 

9 

10 
11 
11 
11 
11 


YES VOTES 


ZERO YES VOTES 


ZERO YES VOTES 


ZERO YES VOTES 


ZERO YES VOTES 


ZERO YES VOTES 




• • 


• • 


• •• 


• • • 


• •• 


Notes: 

The tallies above are reported in pages 1-31 of the Read to Achieve (RtAD) Evaluation Consensus Meeting Notes fronn Novennber 19 
and 20, 2018. The stated purpose of the nneeting is to rank the proposal vendors. 

The nneeting sunnnnary notes that the teann unaninnously agreed to review the "legislatively nnandated specifications" and rank the 
vendors, as follows: "Yes" innplies confornns to specifications; "No" innplies does not confornn to specifications; "Maybe" innplies that the 
teann is unsure about confornnity. 

The chart above shows the "legislated specifications" reviewed at the Consensus Meeting in order of "Yes" votes for Istation. "BS" 
stands for "Business Specification"; "TS" stands for "Technical Specification"; "RMS" stands for "Project Managennent Specification"; 
"RS" stands for "Reporting Specification." 
















































DocuSign Envelope ID: 6DE03F0B-9F25-4589-9F45-C8C87533B6A9 


Statewide IT Procurement Office 
Purchase Transaction Approval (PTA) Form 
For Cancellation 


EXHIBIT B 


Agency: 

Department of Public Instruction 

Tracking Number: 

300042-003 

Agency Requisition No.: 

RQ2068073 

Contract Specialist: 

Andrea Pacyna 


REQUEST APPROVAL FOR CANCELLATION 


Description of Reason for Cancellation: 

DPI is requesting to cancel an RFP for the procurement of Read to Achieve Diagnostics Software. Four (4) offers 
were received on or before the due date specified. All were responsive. DPI subsequently began their evaluation 
and discovered that two (2) of the four (4) Vendors had significant weaknesses as compared to the specifications 
issued in the RFP. Neither Vendor (Curriculum Associates and NWEA) appeared ready to execute a large, state 
wide effort, both had significant technical limitations and compliance with SOC 2 Type II was either absent or 
scheduled to be completed later. 

The two other (2) Vendors that responded appear capable of satisfying our Agency’s business needs but are at a 
disparate price range (so unlike that there is no basis for comparison). Specifically, the prices are: 

Amplify Education: $12,102,096.08 
Istation: $3,098,606.17 

As a result, DPI believes that continuing with the RFP would serve no valuable purpose and they would like to 
cancel. Pursuant to the definition of “NEGOTIATION” in 9 NCAC6B.0316, DPI intends to conduct negotiations with 
sources of supply to maximize the State’s ability to obtain best value based on the evaluation factors set forth in the 
solicitation. 

Contract Specialist Certification: I certify that the above described addendum/amendment was transacted in 
accordance with NC General Statutes, the NO Administrative Code and Policies & Procedures of the Department of 
Information Technology. 



3/25/19 

Contract Specialist - Andrea Pacyna 

Date 


Certification: Statewide IT Procurement Management or Designees: I certify I have reviewed the above described 
amendment/addendum and to my knowledge was transacted in accordance with NC General Statutes, the NC 
Administrative Code and Policies & Procedures of the Department of Information Technology. 


- DocuSigned by: 


-C95C004 1 0F504D5-- 


3/25/2019 I 3:44 FM EDT 


Statewide IT Procurement Manager - Leroy Kodak 


Date 


t - DocuSigned by: 

3/27/2019 1 10:24 

1 D9bi.tsoabiou4h4 Chief IT Procurement Officer - Patti Bowers 

Date 


Rev. 2018/02/26 
























INSTRUCTIONS: Adjust the highlighted text with the appropriate information. Finalize the 
document prior to submitting it to the Statewide IT Procurement Office by deleting this box, the 
directions in red type, and any highlighted areas. Please update the Table of Contents before 
submitting. Note: Per 09 NCAC 06B.0314. completion of specific sections of the template are 
required. These sections are highlighted in blue. 



EXHIBIT C 


Roy Cooper 

Governor 

Eric Boyette 

Secretary of Information Technology 
State Chief Information Officer 


Request for RFP Cancellation 
40-IT00115-18 


To: Patricia Bowers, DSCIO 

Statewide IT Procurement Office 

From: Tymica Dunn 

Date: March 21, 2019 

Subject: Request for RFP Cancellation 

Read to Achieve Diagnostics - Software as a Service 

Reference #: DIT No. 40-IT00115-18 - DPI RFP No. 40-RQ20680730 


Enclosed for your review and approval is a Request for RFP Cancellation for RFP 40-RQ20680730. 

In accordance with 09 NCAC 06B.0316 (Negotiation) and 09 NCAC 06B.0901 (Conditions for Limited or 
Waived Competition), the Department of Public Instruction (DPI) requests approval to conduct 
negotiations with the following two (2) Vendors: Amplify Education, Istation, Curriculum Associates, and 
NWEA. In response to RFP No 40-RQ20680730 DPI received bids from four (4) Vendors prior to the 
submission deadline. 

Discussions regarding vendor evaluations determined that the voting members found the following 
significant weaknesses with the vendors' product: 

Curriculum Associates 

• Fluency measure is not ready and will not be available until the 2021 School Year. 

• There was skepticism about how the foundational questions were assessed just by measuring 
comprehension skills without fluency. 

• The measures for oral language were also very limited. 

• Progress Monitoring appeared to be a shortened version of the assessment and this brings up a 
question about the reliability of the assessment. This service requires a lot of reporting 
customization. It also requires manual intervention for student transfers. 








• The company plans to complete SOC 2 Type II audit in the summer of 2019. 

• The implementation plan proposed is not for statewide implementation and appeared time 
consuming which brought to question the ability to deploy for the 2019 School Year. This 
service will require a lot of customization for a statewide implementation. 

NWEA 

• Service does not directly assess oral language. 

• Progress Monitoring is not in place and implementing this solution will be a problem for the 
2019 School year. Since this service does not have Progress Monitoring in place in the interim 
the growth checklist is used. This tool is also not a good diagnostic and formative assessment 
screener. 

• Their project implementation timeline is not for a statewide implementation and there were 
serious doubts how the statewide implementation can be handled. There were concerns raised 
about teachers viewing other teacher's students. Time must be accounted for customizations to 
be FERPA compliant. 

• The vendor has indicated negotiating state security standards and has not given a clear response 
for SOC 2 Type II audit requirement which will delay approval. 

There are, however, two (2) Vendors that appear capable of satisfying our Agency's business 
needs. Specifically, they are: 

Amplify Education: $12,102,096.08 
Istation: $3,098,606.17 

Soliciting formal offers via IPS again would serve no valuable purpose. Pursuant to the definition of 
"Negotiation" in 9 NCAC6B.0102, DPI intends to conduct negotiations to maximize the State's ability to 
obtain best value based on the evaluation factors set form in the solicitation. We are proceeding in this 
manner. 

DPI would like to formally request that the project remain open in touchdown until a contract is 
established. 

If you have any concerns, please advise. Thank you for your assistance. If additional information is 
required, please do not hesitate to contact us. 


cc: Evaluation Committee 

Leroy Kodak, Contract Manager 
Andrea Pacyna, Contract Manager 
Glenn Poplawski, Chief Solutions Officer 
Kathy Bromead, PMA 



EXHIBIT D 


Evaluation Criteria: Read to Achieve Diagnostics, Requisition No. RQ20680730 


Fall 2018 Evaluation 


Amplify Istation 


Spring 2019 Evaluation 


PRIORITY 

EVALUATION CRITERIA 

RANK 

RANK 

1 

Substantial Conformity To Specification 

1 

2 

2 

RtAD Desired Specifications 

1 

2 

3 

Proof Of Concept / Demonstration 

1 

2 

4 

Vendor Cost Proposal 

4 

1 

5 

Strength Of References 

1 

1 

6 

Vendor Financial Stability 

4 

2 


Overall Rank 

1 

2 


“Personalized Learning" Criterion Listed As “Desirable But Not Required" 


PRIORITY 

EVALUATION CRITERIA 

1 

Vendor Cost Proposal 

—^ 2 

Vendor Financial Stability 

3 

Formative & Diagnostic Assessment 

—► 4 

Personalized Learning 


Source: Fall 2018 Evaluation: Read to Achieve -- 
2018, December 4, 2018 presentation; Spring 2019 
Evaluation: Contract Award Recommendation, June 
7, 2019. 

Note: Evaluation criteria were listed in order of 
importance in both evaluations. 
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EXHIBIT E 


Roy Cooper 

Governor 

Eric Boyette 

Secretary of Information Technology 
State Chief Information Officer 


Contract Award Recommendation 


To: Andrea Pacyna 

Deputy Chief IT Procurement Officer 
Department of Information Technology 

From: Tymica Dunn 

Procurement Chief 
Department of Public Instruction 

Date: May 23, 2019 

Subject: Contract Award Recommendation 

Read to Achieve Diagnostics - Requisition # - RQ20680730, DIT File #300042 

Reference #: Request for Negotiations 40-RQ20680730, DIT File #300042 

Enclosed for your review and approval is the award recommendation for Requisition number 


Bids received pursuant to a RFN #40-RQ20680730A have been reviewed and an Evaluation Committee 
hereby requests the Statewide IT Procurement Office to award the contract, as follows: 


Description: Read to Achieve Diagnostics - Software as a Service 

Recommended Vendor: Imagination Station Inc., dba, Istation 


Cost: 


$8,255,818.50 for 3 years 


Contract Term: Two (2) years plus 1 (one) 

year optional renewals at the discretion of the State 
Project Name and Number: Read to Achieve Diagnostics - 2018 

DIT file # 300042 


Thank you for your assistance. If additional information is required, please do not hesitate to contact me. 

cc: Evaluation Committee 

Patti Bowers, DSCIO 
Glenn Poplawski, DSCIO 
Kathy Bromead, PMA 
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Request for Negotiation 


Section 1: Introduction 

The purpose of this award recommendation and the resulting contract award is to identify a 
vendor best qualified to offer services for Read to Achieve Diagnostic Software as a Service 
solution (RtAD) to meet NCDPI's obligations under state law, N.C.G.S. 115C-83.1, et. seq. 

North Carolina state law requires kindergarten through third grade students to be assessed with 
valid, reliable, formative and diagnostic reading assessments. NCDPI is obligated to adopt and 
provide these developmentally appropriate assessments. The solution must assess student 
progress, diagnose difficulties, inform instruction and remediation, and yield data that can be 
used with the Education Value-Added Assessment System (EVAAS). 

The Evaluation Committee, consisting of four (4) Decision Makers, used "best value" 
methodology authorized by N.C.G.S. §§143-135.9 and 143B-1350(h) during the evaluation 
process. 


Section 2: Evaluation Committee 


Name 

Title/Agency 

Participation 

Levei 

Berry, Erika 

Senior Policy Advisor, NCDPI 

Decision Maker 

Graver, Nathan 

Digital Teaching and Learning Consultant, NCDPI 

Decision Maker 

Karkee, Thakur 

Psychometrician, NCDPI 

Decision Maker 

Shue, Pam 

Deputy Superintendent of Early Education, NCDPI 

Decision Maker 

AlHour, Julien 

Director - Architecture, Integration, & Quality 
Assurance, NCDPI 

SME 

Dunn, Tymica 

Purchasing Section Chief, NCDPI 

Procurement 

Officer 

Gossage, Chloe 

Chief Strategy Officer, NCDPI 

SME 

Strong, Melissa 

NCDPI Attorney 

SME 

Viswanathan, Srirekha 

Project Manager, NCDPI 

Project Manager 
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Request for Negotiation 


Role Definitions: 


Decision Maker: Key business stakeholders evaluating the bid Voting 

responses. 


Project Manager: Overall responsibility includes successful 

initiation, planning, design, execution, 
implementation, and closure of a project. 


Subject Matter Expert Person who is an authority in a particular 
(SME) technical area pertaining to the procurement 


Non-Voting 


Non-Voting 


Section 3: Evaluation Criteria / Methodology 

The selection process was conducted using the "best value" methodology authorized by N.C.G.S. 
§§143-135.9 and 143B-1350(h). The evaluation committee met as a group and evaluated the 
responsive proposals. 

The evaluation criteria listed below is in the order of importance: 


Evaluation Criteria 


Cost 

Vendor Financial Stability 
Formative and Diagnostic Assessment 
Personalized Learning 


Section 4: Timeline 


Date 


Milestone 


March 21, 2019 

RFP Cancellation 


Notifications sent to vendors. Request to Negotiate 


Review Period 

March 27, 2019 

RFP proposals were extended to June 29, 2019 - Clarification 1 

April 11, 2019 

Negotiation Meeting with vendors 

April 17, 2019 

Clarification issued to vendors - Clarification 2 

April 23, 2019 

Clarification response received and shared with evaluation team 

April 25, 2019 

Evaluation Committee meeting and discussion of proposal 
strengths and weakness 

May 3, 2019 

Clarification issued to vendor - Clarification 3 


Clarification response received and shared with evaluation team 

May 15, 2019 

Clarification issued to vendor - Clarification 4 
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Request for Negotiation 



Clarification response received and shared with evaluation team 

May 23, 2019 

Draft Best and Final Offer (BAFO) 

May 23, 2019 

Draft Award Recommendation 


Section 5: Evaluation of Bid Submission 

In response to DIT No. 40-IT00115-18 - DPI RFP No. 40-RQ20680730 four (4) bid proposals were 
received. The RFP cancellation was issued on March 21, 2019 since NCDPI deemed soliciting 
formal offers via IPS again would serve no valuable purpose. Pursuant to the definition of 
"Negotiation" in 9 NCAC6B.0102, DPI conducted negotiations with two (2) of the four (4) 
vendors to maximize the State's ability to obtain best value based on the evaluation factors 
determined appropriate. 


Proposal response from the following two vendors were deemed responsive and considered for 
further negotiations - 


Vendor 

Responsive / Non-Responsive 

Amplify Education Inc. 

Responsive 

Imagination Station Inc. 

Responsive 


Section 6: Vendors 

Listed below is a synopsis of each proposal submitted based on the criteria defined in Section 3. 
A. Evaluation Criteria 

"Best Value" procurement method authorized by N.C.G.S. §§143-135.9 and 143B-1350(h) has 
been used for this evaluation. A one step source selection was used. The proposals were 
objectively evaluated using the evaluation criteria described below. 

The evaluation team members did their due diligence and issued clarifications for each proposal 
before meeting the vendors on April 11, 2019. Strengths and weaknesses were discussed 
during the evaluation meeting on April 25,2019. 

The following evaluation criteria was used to determine strengths and weakness - 

1. Cost 

2. Vendor Financial Stability 

3. Formative and Diagnostic Assessment 

4. Personalized Learning 
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Request for Negotiation 


B. Cost 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 



Cost 


Vendor 

Strengths 

Weakness 

Amplify 

Amplify's clarification response was considered for cost 
evaluation. 

None 

1. The assessment service at $8 per student is expensive. 

2. This assessment does not include lessons. 

3. The assessment requires teacher intervention and is 
not fully online. 

4. Professional Development costs $556,650 for the first 
year and is very expensive. 

5. The expectation from NCDPI Consultants is enormous for 
Training and Professional Development. 

Istation 

Istation's clarification response was considered for cost 
evaluation. 

1. The $5.70 per student option comes with 3000 
teacher directed lessons, access to Istation's 
iPractice to learn and work from home 

2. Students are allowed access from home to all the 

resources. 

3. $76,103 for professional development offers 22 onsite 
training, 14 recorded live webinars and 10 virtual teacher 
training. 

4. Additionally, 5 onsite and 10 recorded webinars will be 
provided at no additional cost. 

None 
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Request for Negotiation 


Cost 

Vendor 

Strengths 

Weakness 


5. The cost for Professional Development also covers 
the logistics and the vendor has taken responsibility 
for logistics. 



After discussing the overall strengths, \A/eaknesses of the cost proposal, the Evaluation Team moved on to the review of 
financial stability. 

C. Vendor Financial Stability 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 


Vendor Financial Stability 

Vendor 

Strengths 

Weakness 

Amplify 

1. There is no going concern. 

1. In a thorough review of the Financial Statement, 

DPI's Financial Director found that the losses have 
gone up from $42,684,000 to $54,459,000. 

2. The financial statement presented was still 
unaudited. No independent auditor's report is 
available. 

Istation 

1. There is no going concern. 

2. This vendor has good liquidity position. 

None 


After discussing the overall strengths, weaknesses of the vendor financial stability, the Evaluation Team moved on to review 
formative and diagnostic assessment capabilities of the vendors. 
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Request for Negotiation 


D. Formative and Diagnostic Assessment 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 


Formative and Diagnostic Assessment 

Vendor 

Strengths 

Weakness 

Amplify 

1. Assessment hits all the mark. 

2. Enough Item pool for 20 assessments. 

3. Good Reporting capabilities. 

1. Assessment is time consuming 

2. Requires excessive teacher involvement in the lower 
cost option 

3. The $8 option is not adaptive 

4. NC has been using Amplify suite for six years now. The 
progress made by students in reading is 
concerning. This brings to question the effectiveness of 
the data driven instructional support. 

Istation 

1. Adaptive assessment allows children to reach their 
full potential. 

2. Excellent reports for teachers to interpret the 
outcome of the assessment. 

3. Enough item pool for 10 assessments 

1. Does not currently support screen readers or keyboard. 


After discussing the overall strengths, weaknesses of the formative and diagnostic assessment capabilities, the Evaluation 
Team moved on to review personalized learning capabilities of the vendors. 


E. Personalized Learning 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 
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Request for Negotiation 


Formative and Diagnostic Assessment 

Vendor 

Strengths 

Weakness 

Amplify 

1. Personalized Learning is good in the Higher option. 

2. Progress Monitoring is available at a subskill level. 

1. The lower cost option does not have all aspects of 
personalized learning and is not adaptive. 

2. Progress Monitoring at the subskill level takes up a lot 
of time for teachers. 

3. Home Reading is not part of the lower cost option. 

Istation 

1. The assessment is adaptive and is online and caters 
to the individual student's need. 

2. The time for assessment is less. This allows teachers 
more time with the students to support their 
individual needs. 

3. Students are allowed to personalize their learning by 
choosing games and activities while working outside 
of school 

None 
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Request for Negotiation 

Based on the review of all evaluation criteria, Istation was ranked first (1^*) and Amplify was ranked 
second (2"'‘). 

The evaluation team unanimously decided to proceed for BAFO with Istation as the only finalist 
based on the overall strengths on the above four evaluation criteria. 


Section?: FinalistVendor(s] 

Based on the consensus decision of the Evaluation Team, Istation was issued the Best And Final 
Offer (BAFO) clarifications. 

Best and Final Offer Negotiations 

The procurement officer of DPI sent the BAFO questions to IStation. 

In the BAFO document the following were clarified and confirmed : 

istation also completed the Vendor Security Assessment Guide (VRAR) that was reviewed by 
DPI and DIT technical teams. 

Section 8: Award Recommendation 

The Evaluation Committee has determined that Istation's bid substantially conforms to the 
requirements of the law, therefore, recommends awarding RFP No. to Imagination Station 
Inc. (istation) in the amount of $$8,255,818.50 for 3 years with the option of one (1) additional 
one (1) year renewals. 

Section 9: Supporting Documentation 

The following supporting documents that reflect the vendor selection are included: 

1. Bid Response - 

2. Clarification documents- 

3. Signed BAFO document 

4. Hosting Exception and Privacy and Threshold Analysis (approved by DIT) 

40-IT00109-17 Rationale for OER Cancellation 
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EXHIBIT F 


SQC 2 Type II Issues 

DPI's request for cancellation also omits the fact that IStation has not yet completed a third 
party security attestation, while highlighting an analogous shortcoming in the proposals from 
Curriculum Associates and NWEA (vendors that were excluded from the negotiation 
process). Notably, public records show that the Evaluation Committee was aware that a failure to 
comply with the third party attestation requirement would make it challenging to obtain DIT's 
approval for an award. 

In DPI's Request for REP Cancellation to DIT, dated March 21, 2019 (Exhibit C), DPI 
stated that "Neither Vendor (Curriculum Associates and NWEA) appeared ready to execute a 
large, state wide ejfort, both had significant technical limitations and compliance with SOC 2 Type 
II was either absent or scheduled to be completed later. In the case of Curriculum Associates, the 
memo states that the company plans to complete SOC 2 Type II audit in the summer of 2019. 

DIT's own cancellation approval then restates this issue identified by DPI in justifying 
excluding the vendors from the negotiation, stating that both Curriculum Associates and NWEA 
"had significant technical limitations and compliance with SOC 2 Type II was either absent or 
scheduled to be completed later." (Exhibit B). 

However, public records show that IStation also has not completed its SOC 2 Type II 
attestation process. The Consensus Meeting notes from November 19, 2018 state that Istation 
“agreed to complete the SOC 2 Type II by Spring 2019” in order to meet the requirement to provide 
a 3rd party attestation (Technical Specification 35). The notes go on to say that "DIT will not 
permit agencies to issue new contracts for moderate level solutions that have not been received an 
acceptable 3rd party attestation. Obtaining approval from DIT to award a contract with a vendor 
that does not already meet this requirement will be challenging." (PDE p. 61 of 166.) 

Public records produced by DIT include a May 22, 2019 email from DPI to DIT, sharing a 
draft BAFO for IStation that includes a roadmap for SOC 2 Type II assessment for the service that 
shows that only the initial readiness assessment will be complete by June 2019, and SOC 2 Type 
II evaluation complete in December 2019, with report available in February 2020. In its initial 
proposal, istation only submitted a SOC 2 Type II report for its hosting provider, Rackspace 
(which is akin to Curriculum Associates referencing an analogous report from AWS — in both 
cases these reports would only extend to the hosting service, not the software application or the 
vendor's other security controls). 

Ultimately, it appears IStation’s proposal suffers from the same SOC 2 Type II issues that 
led to Curriculum Associates and NWEA from being excluded from consideration for award. 



